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SPECTRAL IMAGING OF DEEP TISSUE 
BACKGROUND 

Optical imaging of deep tissue is used to probe structures within biological specimens 
for laboratory research and biomedical purposes. This includes the imaging of internal organs 
and subdermal tissue in animals such as mice, zebrafish, or humans, and one of the goals is to 
learn about internal structures without surgery or other intrusive measures. 

In one technique of deep tissue imaging, fluorescent probes which bind to a specific 
target in the specimen are imaged by exciting them with illumination light, causing them to 
fluoresce; the fluorescent emission is separated from the illumination light, which has a 
different wavelength, by barrier filters and then is detected using a very sensitive camera 
such as a cooled CCD detector. In other techniques, the specimen is infected with agents that 
cause it to produce material that is inherently fluorescent, with the most common example 
being green fluorescent protein (GFP). Further techniques involve use of quantum dots as 
luminous probes. 

As used herein, compounds such as fluorescent probes, GFP, or quantum dots, as well 
as related compounds or others used for similar purposes, are all termed the target 
compounds of a measurement. 

The signals produced in such experiments are typically weak. In general, robust 
detection of the weak levels of light emitted from the deep structures is beneficial because it 
provides earlier, or more reliable, detection of the structures being studied. Also, it may 
enable detection of lower levels of the target compound. Accordingly, techniques or 
apparatus used for deep tissue imaging are valued if they offer a low detection threshold. 

SUMMARY 

The present invention features a method and apparatus to improve detection of target 
compounds such as GFP, quantum dots, and fluorescent probes used for deep tissue imaging 
in a variety of biological samples. Moreover, the method and apparatus are generally 
compatible with use of such compounds, without requiring extreme sensitivity from the 
imaging detector. 



l 



Attorney 12259-034001 
R. Levenson 

The method and apparatus use spectral imaging to distinguish between undesired 
autofluorescence signals arising from various sites in the specimen, and the desired signal 
from the target compound. It has been discovered that the spectral information thus obtained 
provides improved detection sensitivity and can be used for deep tissue imaging, even though 
there is typically significant optical loss in the spectral selection elements of the system. 

In one embodiment, images are taken while viewing the emission light from the 
specimen at a sequence of wavelengths, to develop an image cube with two spatial 
dimensions and a spectrum at each point. By determining the difference in spectral properties 
between the desired target compound emission and the unwanted autofluorescence emission, 
the overall signal is decomposed into components and the detection levels for the desired 
compound emission are greatly improved. 

Other embodiments include measurements based on brightness ratios at several 
selected wavelengths or wavelength bands; measurements based on principal component 
analysis; and measurements based on neural networks, and on fuzzy logic. 

In either case, the method and apparatus use spectral information to distinguish the 
desired signal emitted by the target compound from the unwanted autofluorescence signal, 
and thus to improve the measurement integrity and sensitivity. 

Various aspects and features of the invention will now be summarized. 

In general, in one aspect, the invention features a method including: (i) illuminating a 
sample to cause the sample to emit radiation, wherein the sample includes deep tissue 
supporting a target compound, and wherein the emitted radiation includes emission from the 
target compound and emission from one or more other components in the sample; (ii) 
spectrally filtering the emitted radiation with each of a plurality of different spectral 
weighting functions; (iii) storing an image of the spectrally filtered radiation for each of the 
spectral weighting functions; and (iv) processing the stored images to construct a deep tissue 
image of the sample in which signal from the other compounds is reduced relative to signal 
from the target compound. 

In general, in another aspect, the invention features a method including: (i) providing 
a plurality of images of spectrally filtered radiation emitted from a sample in response to an 
illumination, wherein the sample includes deep tissue supporting a target compound, wherein 
the emitted radiation includes emission from the target compound and emission from one or 
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more other components in the sample, and wherein each image corresponds to a different 
spectral weighting function; and (ii) processing the images of the spectrally filtered radiation 
to construct a deep tissue image of the sample in which signal from the other compounds is 
reduced relative to signal from the target compound. 

5 In general, in yet another aspect, the invention features an apparatus including a 

computer readable medium which stores a program that causes a processor to: (i) receive a 
plurality of images of spectrally filtered radiation emitted from a sample in response to an 
illumination, wherein the sample includes deep tissue supporting a target compound, wherein 
the emitted radiation includes emission from the target compound and emission from one or 

10 more other components in the sample, and wherein each image corresponds to a different 
spectral weighting function; and (ii) process the images of the spectrally filtered radiation to 
construct a deep tissue image of the sample in which signal from the other compounds is 
reduced relative to signal from the target compound. 

In general, in yet another aspect, the invention features an apparatus comprising: (i) a 

1 5 sample holder configured to hold a sample including deep tissue, wherein the deep tissue 
supports a target compound; (ii) an illumination source configured to illuminate the sample 
to cause it to emit radiation, wherein the emitted radiation includes emission from the target 
compound and emission from one or more other components in the sample; (iii) an imaging 
system configured to image the emitted radiation to a detector; (iv) a tunable spectral filter 

20 configured to spectrally filter the emitted radiation with each of a plurality of different 
spectral weighting functions; (v) a detector configured to store an image of the spectrally 
filtered radiation for each of the spectral weighting functions; and (vi) a electronic processor 
configured to process the store images to construct a deep tissue image of the sample in 
which signal from the other compounds is reduced relative to signal from the target 

25 compound. For example, the sample holder may configured to hold an animal, such as a 

mammel, like a mouse, rabbit, or human. Also, for example, the imaging system may have a 
demagnification greater than or equal to 1, and, for example, the imaging system may be 
configured to image a field of view having a diagonal dimension greater than about 2 cm 
onto the detector. 

30 Embodiments of the various aspects of the invention described above may include 

any of the following features. 
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The sample including the deep tissue may be a living organism, such as an animal or 
a mammal. For example, the animal may include a mouse, a rabbit, a zebrafish, or a human. 
Also, the deep tissue may be an internal organ of the living organism, and 

the deep tissue may lie within about 2 mm or more of the living organism. 
5 The deep tissue may be subdermal tissue. 

The emission from the other components of the sample may include autofluorescence 
from tissue overlying the deep tissue. 

The emission from the other components of the sample may include autofluorescence 
from one or more layers of tissue in the sample different from a layer of tissue including the 
10 deep tissue. 

The target compound may be any of, for example, a fluorescent probe bound to at 
least a portion of the deep tissue, a quantum dot bound to at least a portion of the deep tissue, 
a green fluorescent protein (GFP) bound to at least a portion of the deep tissue, a yellow 
fluorescent protein (YFP) bound to at least a portion of the deep tissue, and a red fluorescent 
1 5 protein (RFP) bound to at least a portion of the deep tissue. 

The emission from the target compound may be fluorescence. 

At least some of the spectral weighting functions may correspond to particular 
wavelength bands. For example, all of the spectral weighting functions correspond to 
particular wavelength bands. Alternatively, at least some of the spectral weighting functions 
20 may correspond to sinusoidal weightings of multiple wavelength bands. 

The spectral filtering may include using any of a liquid-crystal, tunable optical filter, 
an interferometric optical filter, and a filter wheel containing a plurality of band pass filters. 

Each stored image may include an intensity value for each of multiple pixels. 

Processing the stored images may include constructing the deep tissue image based 
25 on a weighted superposition of signals in the stored images. 

Processing the recorded images may include constructing the deep tissue image based 
on the recorded images and at least one emission spectrum for the other components in the 
sample. For example, constructing the deep tissue image may include calculating a 
remainder spectrum for each pixel in the set of stored images based on the at least one 
30 emission spectrum for the other components. 
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Similarly, processing the recorded images may include constructing the deep tissue 
image based on the recorded images and an emission spectrum for the target compound. For 
example, constructing the deep tissue image may include calculating a remainder spectrum 
for each pixel in the set of stored images based on the emission spectnim for the target 
compound. 

Also, processing the recorded images may include constructing the deep tissue image 
based on the recorded images, at least one emission spectrum for the other components in the 
sample, and an emission spectrum for the target compound. For example, constructing the 
deep tissue image may include solving at least one component of a matrix equation in which 
one matrix is based on the stored images, and another matrix is based on the emission 
spectra. 

The deep tissue may support multiple target compounds and processing the stored 
images may include constructing a deep tissue image for each of the target compounds. For 
example, processing the recorded images may include constructing the deep tissue images 
based on the recorded images and emission spectra for the target compounds. Furthermore, 
processing the recorded images may include constructing the deep tissue images based on the 
recorded images, the emission spectra for the target compounds, and at least one emission 
spectrum for the other components in the sample. 

The plurality of the different spectral weighting functions may include four or more 
spectral weighting functions. 

Unless otherwise defined, all technical and scientific terms used herein have the same 
meaning as commonly understood by one of ordinary skill in the art to which this invention 
belongs. All publications, patent applications, patents, and other references mentioned herein 
are incorporated by reference in their entirety. In case of conflict, the present specification, 
including definitions, will control. 

Other features, objects, and advantages of the invention will be apparent from the 
following detailed description. 

DESCRIPTION OF DRAWINGS 

The invention will now be further described merely by way of example with 
reference to the accompanying drawings. 
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Figure 1 is an image of a mouse which has been injected with a fluorescent probe, 
imaged at a single spectral band of X = 530 nm. 

Figure 2 is a graph of the emission spectrum of autofluorescence and that of the target 
compound for the sample of Figure 1, with the light-colored line showing the spectrum for 
the autofluorescence and the dark-colored line showing the spectrum for the target 
compound. 

Figure 3 is an image of the target compound emissions from the mouse of Figure 1 , 
with the autofluorescence signal removed using spectral techniques in accordance with the 
present invention. 

Figure 4 is an image of the autofluorescence emissions from the mouse of Figure 1, 
with the target compound emissions separated using spectral techniques in accordance with 
the present invention. 

Figure 5 is a flow-chart of one embodiment of the present invention, based on 
acquisition of a spectral cube and subsequent analysis by linear unmixing. 

Figure 6 is graph of the spectral properties of each spectral band in the embodiment 
of Figure 5. 

Figure 7 is a flow-chart of another embodiment which incorporates PC A to estimate 
the spectra of the autofluorescence and the target compound(s). 

Figure 8 is a flow-chart of yet another embodiment in which the spectral filtering is 
performed interferometrically. 

Figure 9 is a graph of the spectral properties of the spectral weightings used in the 
embodiment of Figure 8. 

Figure 10 is a schematic diagram of a spectral imaging system. 

Like reference symbols in the various drawings indicate like elements. 

DETAILED DESCRIPTION 

The invention features a method and apparatus for reducing the detection level of a 
target compound in deep tissue through spectral discrimination. An important aspect is that 
beneficial result is obtained despite the low light levels involved. 

A schematic diagram of a spectral imaging system 100 for imaging deep tissue is 
shown in Figure 10. System 100 includes a sample holder 1 10 suitable for holding a 
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specimen 1 1 2 having deep tissue. For example, the specimen may be a living organism, such 
as an animal or mammel. A target compound is bound to selected portions of deep tissue in 
the specimen. An illuminator 120 (e.g., a metal halide lamp or other lamp, a laser, an light 
emitting diode array, or any other source of electromagnetic radiation) directs excitation light 
122 to the specimen to excite emission (e.g., fluorescence) from the target compound in the 
deep tissue. Typically, the excitation light will also cause the autofliioresence from the other 
components in the specimen. Therefore, the electromagnetic radiation 130 emitted from the 
specimen includes emission from the target compound as well as autofluorescence. Emitted 
radiation 130 is collected by imaging system 140 and imaged onto a camera 150. 

Because system 100 is designed for imaging deep tissue in relatively large specimens 
(e.g., living organisms), the imaging system typical provides a demagnification of one or 
more, or even 2 or more. That is, the image on the camera is the same size or smaller than 
1 the object field of view for the imaging system. Also, the object field of view for the 
imaging system is typically greater than about 2 cm (or greater than about 3 cm) along a 
diagonal dimension. 

Furthermore, although Figure 10 shows emitted radiation 1 30 as being collected from 
an opposite side of the specimen relative to excitation light 122, in other embodiments, the 
emitted radiation can be collected from the same side as, or at an angle to, that illuminated by 
the excitation light. Moreover, illumination may be provided from multiple sides of the 
specimen- 
Positioned between the specimen and camera 150 is a tunable optical filter 160 (e.g., 
a liquid crystal, tunable optical filter, an interferometric optical filter, or a motorized filter 
wheel). Optical filter 160 spectrally filters emitted radiation 130 according to each of a 
plurality of spectral weighting functions (for example, four or more spectral weighting 
functions). The spectral weighting functions may correspond to specific wavelength bands, 
or may be more complicated functions such as a sinusoid distribution of pass bands. Camera 
150 records images of the spectral filtered emitted radiation 170 for each of the different 
spectral weighting functions, and sends the image data to a computer 180 for analysis. As 
described in greater detail below, the computer processes the image data based on the 
different spectral weighting functions, and one or more emission spectra corresponding to 
pure target compound, pure autofluorescence of one or more other components of the 
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specimen, or both, to construct a deep tissue image that suppresses the autofluorescence 
signal to reveal the target compound. 

In what follows, we describe the context for the spectral imaging of the deep tissue, a 
specific example of deep tissue imaging, and spectral unmixing techniques for constructing 
the deep tissue image. 

It is a hallmark of imaging structures in deep-tissue samples via target compounds 
that the optical signals are relatively weak. Accordingly, many practitioners place prime 
importance on the properties of the detector, and on the efficiency of all elements in the 
optical path, such as the lenses and the blocking filter used to block the excitation light from 
reaching the detector. Yet while the present art of CCD detectors and the like is suitable for 
detecting low light level signals, it does not adequately address the problem of discriminating 
between light emitted by the target compound, and light from other sources such as 
autofluorescence. Thus one's detection level in practice may be set by the level of 
confounding light arising from sites elsewhere within the specimen, rather than 
considerations such as readout noise in one's detector, or the light gathering power of the 
objective. 

Put more precisely, one's detection limit can be seen as the greater of one's detector 
noise or the confounding signal flux which is presented to the detector; expressed in either 
case as the equivalent concentration of target compound in the specimen to produce light of 
that signal level at the detector. 

Unless the light emitted by the target compound dominates over all other sources in 
the specimen, one is often limited by the confounding signal rather than by one's detection 
apparatus/Some level of autofluorescence is inherent in biological samples when they are 
illuminated with light of visible range, especially when the light is green (550 ran) or shorter. 
So despite the use of optimized target compounds, autofluorescence arising at or near the 
specimen surface can often set the detection limit. 

Further, emission from target compounds within deep tissue can be attenuated by 
scattering or absorption as it travels from the site of emission to the surface of the specimen. 
The signal level reaching the imaging system is thus reduced, while light that is generated at 
the surface layer of the specimen is not similarly attenuated. The details of this effect depend 
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on the geometry of the sample specimen relative to the collection optics, as well as the 
optical properties of the sample. 

Likewise, the illumination light may be attenuated or scattered as it travels from the 
source of illumination through the surface layers of the specimen on its way to the structure 

5 being imaged. The excitation signal reaching the target site is reduced, while the signal 
developed at the surface of the specimen is not similarly attenuated. The details of this 
depend on the geometry of the illumination and collection optics, as well as on the optical 
properties of the sample specimen. 

These considerations can exacerbate the effect of autofluorescence by increasing the 

10 relative contribution of autofluorescence emission to the detected signal, compared with 
emission from the target compound. 

The magnitude of the problem is illustrated by Figure I, which is an image of the 
fluorescent emission from a mouse. The mouse was illuminated with light of approximately 
480 nm, and the emission light was filtered by a 25 nm bandpass filter centered at 530 nm. 

15 There is a tumor in the lung of the mouse which expresses the green fluorescent protein 
(GFP). Yet the tumor is not easily distinguishable in the image due to an equal or greater 
signal from generalized autofluorescence, apparently developed in the dermal layers of the 
mouse. As a result, while the signal level at any point in the image is easily quantified, the 
presence of target compound, and thus the tumor, cannot be confirmed due to high levels of 

20 background autofluorescence which sets an equal or higher detection threshold than the 
target compound involved. 

Autofluorescence is also variable from specimen to specimen and can be 
unpredictable. Thus if an absolute flux level is used to make assessments about the target 
compound, one can obtain false positive readings. Variability can arise from factors such as 

25 mold or disease on the skin of the specimen. These are typically not uniform across the 
specimen. So if one seeks to detect the presence ofa target compound by comparing local 
signal levels in a given region against the mean level for the specimen, results are also not 
reliable. 

It is possible in some cases to reduce autofluorescence by choice of the illumination 
30 wavelength. Generally the use of longer wavelengths for illumination is beneficial, as is 
known in the art, since they typically generate less autofluorescence. Also, it can be 
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beneficial to choose a target compound whose emission light occurs at a different wavelength 
range from the autofluorescence of the specimen. Yet it is not possible to choose 
wavelengths that is free of crosstalk, as shown by Figure 2. The emission spectra of the target 
compound, GFP (shown by the darfccolored line), and of the autofluorescence (shown by the 
light-colored line), are completely overlapping. At any wavelength where the target has 
substantial emission, the autofluorescence is also strong, so autofluorescence cannot be 
eliminated by use of a fixed optical filter or something similar. Nor does a color camera 
discriminate between two such similar green spectra. 

Yet as Figure 2 indicates, the spectra of GFP and autofluorescence emissions are 
nonetheless different. Thus, in accordance with the invention, a spectral imaging approach 
can distinguish the two and eliminate or greatly reduce the contribution of the latter signal. 

In a first embodiment of the invention, this is achieved with conventional apparatus to 
illuminate the specimen and to block the illumination light from entering the detector. This 
can be done using an illuminator such as the LT-9500 MSYS from Lighttools Research 
(Encinitas, CA) together with a longpass optical filter that transmits substantially all light X > 
5 1 0 nm, placed in the path of the objective. 

The spectral imaging detector consists of a Qlmaging 1300C digital cooled CCD 
camera (Roper Scientific, Trenton NJ) with a 55 mm F/2.8 Nikkor macro lens (Nikon USA, 
Melville NY) , to which a VARISPEC tunable optical filter (CR1 Inc, Wobum MA) is 
coupled with a mounting adaptor. The VARISPEC filter is a computer-controlled optical 
filter with 25 nm bandpass and tunable center wavelength. These are connected to an IBM 
Thinkpad computer which controls the image acquisition and performs the data analysis. 
Communication is via an IEEE- 1394 interface to the camera, and an RS-232 interface to the 
VARISPEC filter. 

The VARISPEC filter uses nematic liquid crystal variable retarder elements to 
constuct a tunable Lyot filter. The variable retarders are placed in optical series with fixed 
waveplates of quartz or other material, to produce a retardance that is well-known and 
electrically adjustable. Linear polarizers between successive stages of the filter act to block 
unwanted orders so only a single peak is transmitted, and out-of-band leakage can be reduced 
to 0.01% if desired. By choice of the retarder thicknesses, one may obtain bandwidths 



10 



Attorney 12259-034001 
R. Levenson 

ranging from 0.1 nm to 50 run or more. Tuning action is rapid (< 50 ms) and there is no 
image shift from tuning, which is valuable for imaging applications. 

The mouse was imaged by talcing a sequence of images while the center wavelengh of 
the VARISPEC filter was tuned from 500 nm to 650 nm. The result is an image cube, with a 
full two-dimensional image of the sample, and a full spectrum at every point in the image. 
The exact spectrum recorded at a given point depends on the amount of GFP and 
autofluorescence, and on the two spectra, as: 

S(x, y, X) = a(x, y) * ¥(X) + b(x,y) * G(X) [1], 

where (x, y) indices are used to denote a given pixel location in the image, the asterick 
denotes multiplication, X is used to denote a given wavelength of emission or detection, and 

S(x, y, X) denotes the net signal for a given location and wavelength, 
F(X) denotes the emission spectrum of autofluorescence, 
G(X) denotes the emission spectrum of GFP, 

a(x, y) indicates the abundance of autofluorescence at a given (x, y) location, and 
b(x, y) indicates the abundance of GFP at a given (x, y) location. 

Equation [1] states that the net signal from a given location is the sum of two 
contributions, weighted by the relative amount of autofluorescence and GFP present. It is 
easier to see if one writes the above equation for a single pixel: 

S(X) = aFW + bG(A.) [2]. 

F and G may be termed the spectral eigenstates for the system, which are combined in 
various amounts according to the amount of autofluorescence and GFP emission, to produce 
an observed spectrum S. 

Now if the emission spectra of the autofluorescence and of the GFP arc known (or 
can be deduced, as described below), one may invert equation [2] by linear algebra to solve 
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for a and b, provided that the spectrum S has at least two elements in it; i.e. that one has data 
for at least two emission wavelengths X. Then we can write 

A = E" l S [3] f 

where 

A is a column vector with components a and b, and 

E is the matrix whose columns are the spectral eigenstates, namely [F G]. 

Using equation [3] one can take the observed spectrum and calculate the abundance 
of the autofluorescence and of the GFP sources. This process may be repeated for each pixel 
in the image, to produce an image of GFP that is free of contributions from autofluorescence. 
As a result, the detection level is greatly enhanced. 

Note that the matrix E need only be inverted once for a given set of autofluorescence 
and target compound spectra, so the calculation of abundances is not burdensome and can be 
readily done in nearly real-time by a personal computer. 

The results of this process are shown in Figures 3 and 4, which present the abundance 
images for GFP and autofluorescence, respectively. As Figure 3 shows, it is easy to detect the 
GFP once it is separated from the autofluorescence. The degree of improvement in the GFP 
image is striking. Also, one can see that the autofluorescence image is smooth and unaffected 
in the region where GFP is present, which is consistent with the fact that the presence of GFP 
in a deep tissue structure should not alter the amount of autofluorescence emission from the 
overlying dermal regions. 

Overall, the measurement and analysis process is shown as a block diagram in Figure 
5. The specimen is prepared and illuminated (step 505) and the spectral bands to be acquired 
determined (step 510). Then the spectral filter is set to transmit the spectral weighting 
function li, for example, a particular wavelength band (step 515), and an image 
corresponding to that spectral weighting function is acquired (step 520). The spectral filter is 
then set to the next spectral weighting function and the corresponding image acquired until 
all bands have been acquired (steps 525 and 530). The spectra for the target compound and 
the autofluorescence is then provided or otherwise determined (step 535). Based on the 
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spectra, the matrix E is generated and its inverse determined (step 540). For each image 
pixel, the spectra defined by the series of acquired images is then multiplied by the inverse 
matrix of E (step 545) to generate an abundance image of the target compound(s), i.e., the 
deep tissue image (step 550). 

In this example, the invention permitted observation of structures in tissue lying ~2 
mm within a living organism, where the overlying dermis is at least 300 microns thick and 
has significant autofluorescence. The invention has also been used to image structures at 
differing depths in other specimens, including non-mammalian specimens such as zebrafish. 
In the latter, the specimen is physically thinner, but once again there is the problem of 
autofluorescence arising from other layers in the specimen, which confounds the detection of 
target compounds in the interior of the specimen. While there are optical techniques for depth 
sectioning, such as confocal microscopy, the present invention provides a simple and 
practical alternative. 

Nothing about this invention prevents one from viewing multiple target compounds 
per specimen. If we denote the number of spectral settings as n, matrix E becomes annxm 
matrix instead of an n x 2 matrix used in the above example. So, one can use the invention to 
remove autofluorescence from a sample which contains two target compounds; or to remove 
two types of autofluorescence from a sample with one or more target compounds. In any 
case, the result is the isolation of the target compound(s) from the autofluorescence, and the 
ability to quantify one or all of these components. 

The limit to the number of compounds that can be isolated, and to the signal to noise 
generally, is given by the shot noise levels and the degree of spectral distinction between the 
emission spectra of the species being distinguished (including autofluorescence). One can 
describe the degree of correlation between two spectra by an angle 9, defined by 

e = arccos[(SrS 2 )/(|Si||S 2 |)] [4]. 

Sets of spectra for which 0 is small for two members are not as easily separated into 
their components. Physically, the reason for this is easily understood: if two spectra are only 
marginally different, it is harder to determine which species was present, and noise can easily 
change one's estimate of relative abundances. Criteria such as 9 can be used to help decide 
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what spectral bands are appropriate for a measurement, and one may try and select bands that 
yield a large 0 whenever possible. Or, one may make an empirical study of what bands yield 
the best separation, by trial and error. It can be helpful to include more bands than would 
appear necessary from mathematical analysis alone, in order to reduce sensitivity to slight 
spectral shifts from the expected shapes, as may occur due to variation between specimens 
and the like. 

It is worth considering the optical efficiency of the measurement apparatus in the 
above embodiment, to understand where the inventive improvement comes from. First, the 
lens used was an F/2.8 type instead of an F/1.2 or F/1.8 which is.more typical for this work, 
and this choice results in 2,4 - 5.4x less light collection. Next, the VAR1SPEC filter has a . 
transmission of approximately 25 percent, and collects over a 25 nm range, in contrast to a 
typical interference filter which has a transmission of 80 percent and collects over a 40 nm 
range. This further reduces the sensitivity by a factor of 5.1 x compared to equipment that 
might be used for this work, for an overall reduction in light flux of 12.3x - 27.8x compared 
to the best practice alternatives of the art. 

The CCD camera is cooled 25° below ambient to approximately 0°C, which is typical 
for an ordinary CCD sensor, unlike the sensors used in imaging stations such as the 
ChemiPro system from Roper Scientific (Trenton, NJ)., which is cooled with liquid nitrogen 
to attain temperatures 1 00° below ambient or lower. 

As this suggests, the effectiveness of this technique does not arise from extreme 
efficiency in the gathering or collection of light; rather it comes from using spectral 
selectivity to identify and remove the effects of background fluorescence. 

Turning now to the question of how the spectra F and G are determined, any method 
may be used which yields an adequate estimate of the spectra involved. For some target 
compounds, there is a known spectrum for the material from published references. 
Alternatively, with a spectral imaging station as is used in the invention, one may obtain the 
spectrum directly by placing a sample containing a sufficient concentration of the target 
compound in front of the imager and taking its spectrum. Conversely, it is often possible to 
image a region of the specimen where one has a priori knowledge that there is no target 
compound in that region, and in this way one can obtain an estimate of that component. 
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Various data analysis techniques can be used in this process, such as principal 
component analysis (PCA), which identifies the most orthogonal spectral eigenvectors from 
an image cube, and yields score images showing the weighting of each eigenvector 
throughout the image. If PCA analysis is performed on an image that contains contributions 
from the target compound(s) and from the background autofluorescence, the vectors from 
PCA can be used to develop estimates of the spectra involved. 

This may be done in combination with other mathematical processing, and there are 
other known techniques for identifying low-dimensionality spectral vectors, such as 
projection pursuit, a technique described in L. Jimenez and D. Landgrebe, "Hyperspectral 
Data Analysis and Feature Reduction Via Projection Pursuit", IEEE Transactions on 
Geoscience and Remote Sensing. Vol. 37, No. 6, pp. 2653-2667, November 1999. 

An embodiment that incorporates PCA is shown in Figure 7 in block-diagram form. 
The block diagram is similar to that of Figure 5, except that step 535 is replaced with 
performing a PCA analysis (step 735) and determining eigenvectors corresponding to the 
autofluorescence and target compound(s) (step 737) for use in the matrix E in step 540. 

Whatever method is used in the determination, the goal is to derive spectra 
corresponding to the species sought, whether the determination is made by a direct 
measurement of relatively pure samples of the target compound and the autofluorescence, or 
a mathematical analysis of spectra that may contain mixtures of both components, so long as 
adequate spectral estimates are obtained. 

While the embodiments discussed above use spectral estimates for both the 
autofluorescence emissions and target compound emissions, this is not essential. In some 
cases, one may seek to reduce the autofluorescence without having a priori knowledge of the 
target compound spectrum. This can be done by looking at the signal in a given pixel, and 
subtracting from it the maximum amount of autofluorescence while leaving the remaining 
signal that is positive definite in all spectral channels. That is, one defines a so-called 
remainder spectrum R a (X) for each pixel: 

R a (X) = S(X)-aF(X) [5a], 

and then selects the largest value of parameter a consistent with RJl) having a non-negative 
value in every spectral channel. The resulting spectrum Ra(X) is then used as the sample 
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spectrum, expunged of autofluorescence. One may also make the determination of a based 
not on strict non-negative criterion listed above, but on some related criteria that incorporates 
a small negative distribution, to account for considerations such as shot noise or detector . 
noise. 

Alternatively, one may seek to determine the distribution of the target compound by a 
similar method when its emission spectrum is known, but the autofluorescence spectrum is 
not, by seeking to subtract off from S(X) the maximum amount of target emission GQC) 
consistent with a positive remainder, and then reporting the amount that was subtracted at 
each pixel as the image of the target compound. In this case, the remainder spectrum R^(k) 
for each pixel is given by: 

Rb(X) = SW-bG(*) [ 5b l> 

where one selects the largest value of parameter b consistent with R^fl) having a non- 
negative value in every spectral channel. 

Furthermore, the technique described above in connection with Equations 5a and 5b 
can be expanded to cases where the spectra for one or more additional components of the 
sample are known, and one wants to remove their contribution to the signal. In such cases 
the remainder spectrum is rewritten to subtract a contribution of each such component from 
the observed signal based on the additional spectra and consistent with a positive remainder 
in each spectral channel. 

Another preferred embodiment uses the same apparatus to illuminate-and view the 
specimen, except that the specimen is a zebrafish in an aqueous sample stage. 

Yet another alternative uses the same apparatus to view a mouse that has been 
transfected to express either the yellow fluorescent protein (YFP) or the red fluorescent 
protein (RFP), or both, and produces images of the target compound(s) after removal of the 
autofluorescence signal. There are also mutant strains developed which may also be used. 
Any of these may be combined with the GFP when that produces useful results. 

Further alternative embodiments view mice containing target compounds based on 
quantum dots, and incorporate an illuminator and filter set optimized for the quantum dot 
species involved. 
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An embodiment operating in the infrared range 600 - 1 100 nm may also be 
constructed using a near-infrared VAR1SPEC filter such as the model VIS-NIR2-10-20HC. 

In an alternative embodiment, the VariSpec filter is replaced with a motorized filter 
wheel containing a plurality of bandpass filters. Yet another embodiment uses a split-image 
system from Optical Insights (Tucson, AZ) to view the specimen in four spectral bands at 
once, albeit with lower spatial resolution. The bands are chosen to give a spectrum that 
distinguishes between the target compound and background autofluorescence, i.e. to have cos 
9 that is significantly different from I, preferably 0.8 or less. 

It is not necessary that the images used for the spectral analysis be acquired with 
bandpass filters, only that the spectral weightings of the various images be different. For 
example, one could use an interferometer to acquire the spectral information, as shown in 
Figure 8 in block-diagram form. The spectral response of the interferometer is shown in 
Figure 9 for some selected values of path difference Z. Images thus obtained can be used for 
practicing the invention, either directly or after transforming from interferograms to spectra. 
The suitability of using the interferograms can be checked by looking at how well they 
distinguish between the species involved, which can be determined by measures such as cos 
6 or by experimental study. 

The block diagram of Figure 8 is similar to that of Figure 5 except that: steps 515, 
520, 525, and 530 are replaced with corresponding steps 815, 820, 825, and 830, which use a 
mterferogram as the spectral weighting function rather than particular spectral bands; there is 
an optional step 832, which describe Fourier transforming the spectrally filtered images to 
generate a spectral cube; and step 535 is replaced with step 835 determines spectra or 
interferogram weightings for the target compound and the autofluorescence consistent with 
the form of the acquired data (and optional Fourier transform) for use in generating the 
matrix E. 

The interferometer can be a mechanical type such as a Sagnac design, or it can be a 
birefringent interferometer as described in U.S. Patent 6,421,131, "Birefringent 
interferometer". The latter uses fixed retarder elements such as quartz plates, together with 
switching apparatus, to make the retarders add or cancel one another, so that using these 
elements, along with variable retarder elements, one can produce any desired retardance 
within a wide range. When polarized light encounters this assembly, its polarization state is 
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changed in a manner that depends on the wavelength of light, and this can be detected at an 
exit analyzer polarizer. The spectral response at any particular setting of the interferometer is 
a sinusoid in 1A,, after allowing for dispersion. By taking a sequence of readings at known 
retardance values, and performing a fourier transform, the spectrum of the light can be 
determined. Such apparatus can be used in imaging systems to obtain a spectrum at every 
point in an image, or simply to obtain a set of images with various sinusoidal spectral 
response functions in the members of the set. 

Indeed, any spectral imaging apparatus can be used provided that it yields adequate 
spectral information to distinguish emission by the target compound from background 
autofluorescence. 

The spectral analysis and construction of the deep tissue image can be implemented 
in hardware or software, or a combination of both. The electronic processing can be 
implemented in computer programs using standard programming techniques following the 
methods described herein. Program code is applied to input data to perform the spectral 
unmixing functions described herein and generate output information such as the deep tissue 
image. The output information is applied to one or more output devices such as a display 
monitor. 

Each program is preferably implemented in a high level procedural or object oriented 
programming language to communicate with a computer system. However, the programs 
can be implemented in assembly or machine language, if desired. In any case, the language 
can be a compiled or interpreted language. Moreover, the program can run on dedicated 
integrated circuits preprogrammed for that purpose. 

Each such computer program is preferably stored on a storage medium or device 
(e.g., CD-ROM or magnetic diskette) readable by a general or special purpose programmable 
computer, for configuring and operating the computer when the storage media or device is 
read by the computer to perform the procedures described herein. The computer program 
can also reside in cache or main memory during program execution. For example, computer 
180 in Figure 10 may includes a processor, an input/output control card, a user interface, 
such as a keyboard and monitor, and a memory. A program stored on a computer-readable 
medium is stored in the computer memory, and when executed, the program causes the 
processor to carry out the steps of analyzing the spectrally filtered images. 
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While certain embodiments have been described, alternatives may be used that 
achieve the same end without deviating from the spirit of the invention, according to the 
requirements of the task at hand and normal design considerations. It is explicitly intended 
that this invention can be combined with the arts of instrument design, multispectral data 
analysis and image analysis, and optical system design, as appropriate for a given use. 

Additional aspects, features, and advantages are within the scope of the following 

claims. 
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WHAT IS CLAIMED IS: 

1 . A method comprising: 

illuminating a sample to cause the sample to emit radiation, wherein the sample 
comprises deep tissue supporting a target compound, and wherein the emitted radiation 
comprises emission from the target compound and emission from one or more other 
components in the sample; 

spectrally filtering the emitted radiation with each of a plurality of different spectral 
weighting functions; 

storing an image of the spectrally filtered radiation for each of the spectral weighting 
functions; and 

processing the stored images to construct a deep tissue image of the sample in which 
signal from the other compounds is reduced relative to signal from the target compound. 

2. The method of claim 1, wherein the sample comprising the deep tissue is a 
living organism. 

3. The method of claim 2, wherein the living organism is an animal. 

4. The method of claim 3, wherein the living organism is a mammal. 

5. The method of claim 3, wherein the animal comprises a mouse, a zebrafish, or 
a human. 

6. The method of claim 2, wherein the deep tissue is an internal organ of the 
living organism. 

7. The method of claim 2, wherein the deep tissue lies within about 2 mm or 
more of the living organism. 

8. The method of claim 1 , wherein the deep tissue is subdermal tissue. 
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9. The method of claim I , wherein the emission from the other components of 
the sample comprises autofluorescence from tissue overlying the deep tissue. 

10. The method of claim I, wherein the emission from the other components of 
the sample comprises autofluorescence from one or more layers of tissue in the sample 
different from a layer of tissue comprising the deep tissue. 

11. The method of claim 1 , wherein the target compound is a fluorescent probe 
bound to at least a portion of the deep tissue. 

12. The method of claim I, wherein the target compound is a quantum dot bound 
to at least a portion of the deep tissue. 

13. The method of claim 1 , wherein the target compound is a green fluorescent 
protein (GFP) bound to at least a portion of the deep tissue. 

14. The method of claim 1, wherein the target compound is a yellow fluorescent 
protein (YFP) bound to at least a portion of the deep tissue. 

15. The method of claim 1, wherein the target compound is a red fluorescent 
protein (RFP) bound to at least a portion of the deep tissue. 

16. The method of claim 1 , wherein the emission from the target compound is 
fluorescence. 

3 7. The method of claim 1, wherein at least some of the spectral weighting 
functions correspond to particular wavelength bands. 

18. The method of claim 17, wherein all of the spectral weighting functions 
correspond to particular wavelength bands. 
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19. The method of claim 1 , wherein at least some of the spectral weighting 
functions correspond to sinusoidal weightings of multiple wavelength bands. 

20. The method of claim 1, wherein the spectral filtering comprises using a liquid- 
crystal, tunable optical filter. 

2 1 . The method of claim 1 , wherein the spectral filtering comprises using an 
interferometric optical filter. 

22. The method of claim 1 , wherein the spectral filtering comprises using a filter 
wheel containing a plurality of band pass filters. 

23. The method of claim I, wherein each stored image comprises an intensity 
value for each of multiple pixels. 

24. The method of claim 1 , wherein processing the stored images comprises 
constructing the deep tissue image based on a weighted superposition of signals in the stored 
images. 

25. The method of claim 1 ; wherein processing the recorded images comprises 
constructing the deep tissue image based on the recorded images and at least one emission 
spectrum for the other components in the sample. 

26. The method of claim 25, wherein constructing the deep tissue image 
comprises calculating a remainder spectrum for each pixel in the set of stored images. 

27. The method of claim 25, wherein processing the recorded images comprises 
constructing the deep tissue image based on the recorded images, the at least one emission 
spectrum for the other components in the sample, and an emission spectrum for the target 
compound. 
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28. The method of claim 27, wherein constructing the deep tissue image 
comprises solving at least one component of a matrix equation in which one matrix is based 
on the stored images, and another matrix is based on the emission spectra. 

29. The method of claim 1 , wherein processing the recorded images comprises 
constructing the deep tissue image based on the recorded images and an emission spectrum 
for the target compound. 

30. The method of claim 25, wherein constructing the deep tissue image 
comprises calculating a remainder spectrum for each pixel in the set of stored images. 

3 1 . The method of claim 1 , wherein the deep tissue supports multiple target 
compounds and processing the stored images comprises constructing a deep tissue image for 
each of the target compounds. 

32. The method of claim 31 , wherein processing the recorded images comprises 
constructing the deep tissue images based on the recorded images and emission spectra for 
the target compounds. 

33. The method of claim 32, wherein processing the recorded images comprises 
constructing the deep tissue images based on the recorded images, the emission spectra for 
the target compounds, and at least one emission spectrum for the other components in the 
sample. 

34. The method of claim 1 , wherein the plurality of the different spectral 
weighting functions comprises at least four spectral weighting functions. 

35. A method comprising: 

providing a plurality of images of spectrally filtered radiation emitted from a sample 
in response to an illumination, 
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wherein the sample comprises deep tissue supporting a target compound, 
wherein the emitted radiation comprises emission from the target compound and 

emission from one or more other components in the sample, and 

wherein each image corresponds to a different spectral weighting function; and 
processing the images of the spectrally filtered radiation to construct a deep tissue 

image of the sample in which signal from the other compounds is reduced relative to signal 

from the target compound. 

36. Apparatus comprising a computer readable medium which stores a program 
that causes a processor to: 

receive a plurality of images of spectrally filtered radiation emitted from a sample in 
response to an illumination, 

wherein the sample comprises deep tissue supporting a target compound, 

wherein the emitted radiation comprises emission from the target compound and 
emission from one or more other components in the sample, and 

wherein each image corresponds to a different spectral weighting function; and 
process the images of the spectrally filtered radiation to construct a deep tissue image of the 
sample in which signal from the other compounds is reduced relative to signal from the target 
compound. 

37. An apparatus comprising: 

a sample holder configured to hold a sample comprising deep tissue, wherein the deep 
tissue supports a target compound; 

an illumination source configured to illuminate the sample to cause it to emit 
radiation, wherein the emitted radiation comprises emission from the target compound and 
emission from one or more other components in the sample; 

an imaging system configured to image the emitted radiation to a detector; 

a tunable spectral filter configured to spectrally filter the emitted radiation with each 
of a plurality of different spectral weighting functions; . 

a detector configured to store an image of the spectrally filtered radiation for each of 
the spectral weighting functions; and 

24 



Attorney 12259-034001 
R. Levenson 

a electronic processor configured to process the store images to construct a deep 
tissue image of the sample in which signal from the other compounds is reduced relative to 
signal from the target compound. 

5 38. The apparatus of claim 37, wherein the sample holder is configured to hold an 

animal. 

39. The apparatus of claim 37, wherein the imaging system has a demagnification 
greater than or equal to 1 . 

10 

40. The apparatus of claim 37, wherein the imaging system is configured to image 
a field of view having a diagonal dimension greater than about 2 cm onto the detector. 
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ABSTRACT 

Apparatus and methods are provided for the imaging of structures in deep tissue 
within biological specimens, using spectral imaging to provide highly sensitive detection. By 
acquiring data that provides a plurality of images of the sample with different spectral 
weightings, and subsequent spectral analysis, light emission from a target compound is 
separated from autofluorescence in the sample. With the autofluorescence reduced or 
eliminated, an improved measurement of the target compound is obtained. 
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